4» python" for black-box testers 

»> lenC'Sajnikanth Suriyanarayanan") 



About me 



Coastal GeoScientist and Civil Engineer 
10+ years in Software Quality Assurance 
and Functional Testing 
HP Certified Professional in Quality Center 
Lead Quality Assurance at Vistaprint 
Contributor to Holmium Core ^^^^^^ 
http://sajnikanth.com BBP ^HB 



Contents 



• Background to black-box testing 

• Manual vs Automated testing 

• Why python? 

• selenium and other uses 

• Lessons Learned 



$ python 



m 



^^^^^^^^^ 




Black-box tester? 




Black-box tester's task list 



• Prepare test plans and test cases 

• Test the application 

• Find and record defects 

• Re-test 

• Regression test 




Source - http://innovationJv/ino2/publications/leonardo_manual/en/www Jnnosupport.net/webhelp/wso/index.cfm 



Qualities of a good black-box tester 
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What's the 




How many test cases do you see? 



5*3*3*2*2 = 180 

Test Cases * Browsers * Countries * Languages * Environment 



Time to test on a few platforms? 
Good luck sucker!! 



What's the problem? 



Repetitive manual regression = 
Low test coverage 
No common language 
Ever growing Test Suites 
Testing becomes a bottleneck 
QA bites the bullet ■H^H 



boring 
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Source - 



http://en.wikipedia.org/wiki/lnfinite_monkey_theorem 



Automated testing 



• "Fail fast; fail often" - Martin Fowler 

• Integration / Functional / Performance 

• Building block for continuous integration and 
deployment 

• Higher Return on investment 



QTP to the rescue 

• Familiar (HP Certified after all) 

• Popular 

• Supports Record and Play 

• Expensive 

• Proprietary 

• Windows-centric 

• "Ageing" community 

I think it is time for QTP to die - http://paulhammant.com/blog/die-qtp-die.html/ 




python? 



• Easy to setup 

• Readable code (easy for beginners) 

o forced indentations; better for the team 
o consistency 

• open source; active community / library 



»> from selenium import webdriver 
»> firefox = webdriver. Fir efoxC) 
»> f i ref ox . get C " http : //googl e . com " ) 

»> firefox. find_element_by_idC"gbqfq"3.send_keysC"Hello World") 




back to the problem 



5*3*3*2*2 = 180 

Test Cases * Browsers * Countries * Languages * Environment 



»> from selenium import webd river 
»> for browser in ("firefox", "chrome", "phantomjs"): 
for countries in C"NT", "ID", "IN"): 
for languages in ("EN", "ID"): 

for environments in ("develop", "staging"): 
def test_lC): 

.teps 
def test_20: 



Time to test on a few platforms? 



Selenium Features 



• Record and Play (Firefox) 

• Multiple browsers 

• Mobile Support 

• Multiple ways to locate an element 

• Screenshots 




Working with python 

• Test APIs 

• Write scripts for simple Ul tests 

• Create test data 

• Test Suites 

• Performance Testing 



$ python 



Lessons learned 



Lessons learned 



»> from selenium import webdriver 

»> firefox - webdriver.FirefoxC) 

»> chrome ■ webdriver.ChromeC) 

»> phantom js - webdriver.PhontomJSC) 

>» for browser in C'Pir'efox, chrome, phantom js): 

b rowse r « get C " htt p ://nt. vi s tamobi . com/ nt/en/s/dashboa rd " ) 

, , , b rowse r . f i nd_el ement_by_idC " i nt rologi n_name " ) , send_keysC " use r name " ) 

browser . f ind_element_by_idC"intrologin_password") . send^keysC" password") 
browser. find_element_by_idC"validate-login").clickC) 



Web dev changes conventions - intrologin-name, 

intrologin-password etc. 



Lessons learned 




1 fPQin holmiun.cope import PageObject, PogeElement, Locators 
2 

% class LoglnMainCPi^eObject); 



□serniQine_lMX = PogeElanentCLocators^ID, "introlagirr_nane", tiineotitsZa) 
fMss«or[i_box = PageElementCLocators.ID, "irrtrologinjpassword", tlmeoiita28) 
signal OLitton = PogeElefnentCLocators.ID, "validate-logln", tlmeoLit^za) 
register_llnk - PogeElanerrtCLocators.LlNICTEXT, "Register here'', timeoLrta20^ 



def do_einail_loglnCself , user none, password); 
sel f , user riane_tKix , clear CD 
sel f , user riane_box . send-key sCuser none J 
sel f . fMsswor cLImx . clear CD 
sel f , passwor cLImx ^ send_lt^sCpass«ordD 
sel f , si grui rr_:bLitton , cl i ckCD 



1 . Use Page Objects 

PageObjects by Adam Goucher - 
http://www.pushtotest.com/create-robust-selenium-tests-with-pageobjects 



def test_loginCself): 

login_page = pages. login. LoginMainCself .driver, self .dashboard_url) 
login_page . do_email_loginCself . username , self . password) 
dashboard_page = pages. dashboard. DashboardMainCself .driver) 
assert dashboard_page . account_button . is_displayedC) 



Lessons learned 



1 from holmiun.core import PageObject, PageElanent, Locators 

2 irport time 
3 

4 class LoginMainCPogeObject); 

5 

6 u5ernarie_box = PageElemerrtCLocators.IO, "intrologin_naie", timeout=Zl0) 

7 passworcl^box = PageElemenrtCLocators.ID, "intrologirx^Mssword", timeout-Z!0> 

5 sigruinJiLrtton - PageElanerTtCLocators,ID, "validate-login", timeout-za) 

9 register_link = PageElanentCLocator^,LINK_TEXT, "Register here", timeoiits20) 



def do_email_loginCself , usernane, password); 
tirae,sleepC5) 
set f . user none-box , clear CD 
se L t . user name jtx3x . seria-Keysi;^user name j 
sel f , password-box . cl ear C) 
self , password^box. send-keysCpasswordD 
sel f - sign_i nJjutton , cl i ckC> 



Lessons learned 



1 from lialmiLjn.care import PageObjIect, PogeElanent, Locators 
2 

B class LoginMainCPageOjject); 
4 

5 usernanejbox = PageElanerrtCLocators.ia, "intpologirunoiie", tiineoiJt=Z!a> 

6 password-box = PageElanerrtCLocators.ia, "lritrologin_passwDrd", tim&KsbsW) 

7 sigrT_irTjxjt±on = PageElementCLocotors-IQ, "validate-login", tiineoutsZa) 

S register.llnk = PageEleinerTtCLocators.LINie.TEXT, "Register here", timeout-Zft) 

9 

L0 def do_einall_loginiCself , usernone, password); 

LI set f . driver , irpL i citly.wai tC2!0) 

L2 I sel f . user naneJjox . dear C> 

L3 sel f . user nflPie_box . send_k^sCuser none) 

14 sel f . [MsswortLbox . cl ear C) 

15 sel f . password-box . serid-keysCpassword> 

16 self ,5igri_inJxjt±on.clickQ 

2. Wait instead of Sleep 



Explicit and Implicit Waits - 
http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp 



Lessons learned 

3. pick mgjg lgj 

• Organize tests better 

• Run selected tests 

• Plugin Architecture 

• List all tests 

• Test Runner (less boilerplate code) 



nose is nicer testing for python - 
https://nose.readthedocs.org/en/latest/ 



Lessons learned 



• Run 30 tests on Firefox 

o about 12 minutes 



• Run 30 tests on a Headless browser 

o about 7 minutes 



Lessons learned 



4. use headless 

• step toward Continuous Integration 

o Jenkins / teamcity / Travis CI are pliantomjs-ready 

• better performance than Firefox / Chrome 

• works without GUI 

o no interruption 
o ec2 micro instance 
o vagrant 




Introducing Holmium 



https://holmiumcore.readthedocs.org 
plug in for nosetests 

o developed by Ali-Akber Saifee 
■ http://ali.indydevs.org/ 



Page object definitions made easy 
Other options: 

o Specify environment 
o Select browsers 
o Run tests remotely 



HO 
LM 



I jfpoin iTolmiuffi.oore import PogeObject, P^^eElemerrt, Loootors | 

3 class LoginMainCPogeObject); 

4 I I 

5 usernane^box = PageEleffientCLocotors.ID, "inrtpol jginunopie", timeaut5=20> 

6 passwortLbox - PogeEleinentCLocators.ID, "irrtrol >gin(_pass«ord", tlmeoutaZfl) 

7 sign_inJbtJt±on ^ PogeEleinentCLocators-IQ, "vali iote-login", timeout-ZO) 

S register_link ? PageElemerttCLocotorsAINICTEXT, "Roister liere", tlmeoLtt-ZO) 



def do.anall^loglnCself , usernoue, password); 
sel f , driver , impl 1 ci tly_i«ai t C23) 
sel f , user mneJxnc , clearO 
sel f , user riane_bax . send-key sCuser none) 
sel f . password-bflx . cl earC) 
sel f . password-box . send-key sCpassword) 
sel f , si gn_i nJbutton , cl i ckQ 



nosetests tests/vistannobi_tests.py 

-with-holmium 

-holnniunn-browser=firefox 

-holnniunn-environnnent=staging 

-holnniunn-useragent=iphone 



How many software testers does it take to change a lightbulb? 
None; testers just report that the room is dark. 

Questions? 



View this presentation here - 
http://j.mp/python-for-black-box-testers 



